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METHOD AND APPARATUS FOR server becomes overloaded as it attempts to serve a large 

DISTRIBUTING REQUESTS AMONG A number of requests for many different documents. An 

PLURALITY OF RESOURCES example of this kind of server is a server for a large 

organization, such as a library, that serves a vast number of 
GOVERNMENT SUPPORT 5 documents that are accessed on a regular basis. 

Work described herein was supported by Federal Grant \^ t gCD ° ral Strat ° gy cm P lo y cd to rcducc both nctwork 

No. DAAH04-95-10607, awarded by the United States I ■ traffic and overloaded servers is replication accomplished by 

Army, Federal Grant No. N0001 4-95-1 -1246, awarded by I f cacl ™S- Copies of frequently requested documents can be 

the United States Navy, and CCR-9624239, awarded by the ! st r orcd throughout the network, thereby spreading the work 

National Science Foundation. The Government has certain 1° of mc documents across several servers. In one 

rights in the invention approach, several clients share a cache server in a manner 

I that is transparent to the people using it. Client requests are 

FIELD OF THE INVENTION forwarded through the cache server, which keeps copies of 

I frequently requested documents and attempts to satisfy 

The invention relates to allocating resource requests to the / 15 requests with a cached copy. If the cache server has a copy 

available resources. More specifically, the invention relates f of the document, then the request can be answered imme- 

to distributing information requests to servers capable of! diately with the requested document. If the cache server does 

storing and providing the requested information. j r\ 0 t have a copy, then the cache server can request a copy of 

^ A mrn „ „ „ „ \ the document from the original site. When the cache server 

BACKGROUND OF THE INVENTION ) 2 o ^ ccives a copy of the document from another cache server 

In large distributed information systems there are servers ! °f fr° m tne original site for the document, it can then answer 

that store information and there are clients that request ^ c chant's request Subsequent requests for that document 

information. The World Wide Web (" Web") is an example of | can bc serviced by the cache because it has a copy of the 

a distributed information system. There are web sites, which ! document. Thus, the load on the original site for the docu- 
act as servers, and there are clients, which are computers ? S J^ cnt ^ re ^ uccd because the original site does not need to be 

used by people to obtain information from the servers. The jj involved in every request. 

users request information from the web sites and the web Referring to FIG. 1A, three clients, Client One 110, Client 

sites respond to the requests by providing the requested Two 111, and Client Three 112, each request documents 

information in the form of data, documents, files, or web Docl, Doc2, and Doc3 respectively via Cache Server A 100. 

pages. 30 The cache server answers all three of these requests in a 

Immense growth of the Web places great demands on the manner that is transparent to the users. If Server A 100 does 

infrastructure of the Internet that provides the physical not nave me documents requested by Client One 110, Client 

connections for the Web. Service providers invest vast sums Two HI, ^d Client Three 112, it requests the documents 

of money in Internet infrastructure in an attempt to provide 35 froEa toe Original Sites 102, 104 for the documents. Once it 

acceptable performance despite increasing data demand. receives the documents it sends the documents to the Clients 

Acceptable performance generally implies little or no delay 110-112. 

in accessing information. Two causes of delay are heavy Caching can reduce network traffic if the cached copy is 
communication loading a part of a network and a large close, in the network topology sense, to the requester 
number of requests loading a particular server. When part of ^ because fewer network links and resources are used to 
the network becomes congested with too much traffic, retrieve the information. Caching can also relieve an over- 
communication over that part of the network becomes loaded server because some of the requests that would 
unreliable and slow. Similarly, when too many requests are normally be routed to the original site can be served by a 
directed at a single server, the server becomes overloaded, or cache server, and so the number of requests made of the 
"swamped." 45 original site is reduced. There is a tradeoff of a greater 

One of these problems can cause the other, in part because performance benefit if more clients share the same cache 

network software is typically designed to repeat requests server that is balanced against an increased likelihood that 

that are not responded to promptly. For example, if a server ^e cache server itself will become overloaded, 

is overloaded, it will not respond to requests quickly, and One approach to solving this dilemma is to use a group of 

requests may be resent, thereby increasing network traffic. If 50 servers that function together as a cache. Responsibility for 

a nctwork is overloaded, a request may not reach a server answering requests is divided among the group. There are 

quickly, and a duplicate request may be sent to the server many ways to divide responsibility among the group of 

before the server had a chance to respond to the first request, servers. One way is to have a client's request for a document 

thereby increasing the load on the server. Both network be directed to an arbitrary one of the servers of the group of 

overload and server overload lead to delays in retrieving 55 cache servers. If a copy of the document is stored in that 

information. server, it is transmitted to the client. If a copy is not stored 

A server can become overloaded for various reasons. A m ^at server, the request is forwarded to the original site of 

server storing a few documents that have very high access mc document. This approach divides the load among the 

rates can become overloaded. An example of this kind of servers, thereby preventing overload of any particular server, 

server is a web site that becomes very popular such as a site 60 A clicnt needs to know which servers are available, but the 

reporting results of a popular sports event. A server respon- client does not need information about the servers to choose 

sible for such a site can become overloaded with a large onc - 0 QC disadvantage of this approach is that each of the 

volume of requests for a one or a small number of docu- servers will request each document from the original site for 

ments containing the latest information on the event. mat document. This could load the original site as the 

Another type of server that can become overloaded is a 65 number of cache servers increases. 

server storing a large number of documents each of which is Alternatively, the server could, if it does not have a copy 

accessed at normal or even relatively low access rates. The of the document, first forward the request to the other cache 
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servers. One disadvantage of this technique is that as the One disadvantage of using a standard bash function to 

number of participating servers grows, the number of mes- allocate documents, and, therefore, documents requests, to 

sages between the cache servers can become problematic. servers is that a change in the number of servers can be 

Another approach also uses a group of servers but alio- disruptive to the caching scheme. In networks such as the 

cates responsibility for documents to particular servers. As 5 Internet, servers frequently may be added to, or removed 

part of this approach, the client is informed about the from, the network, and these changes need to be incorpo- 

allocation of documents to servers so it can address a request rated into a caching scheme. With the hash function just 

to the server with responsibility for that document. Instead discussed, adding a server to the scheme requires changing 

of arbitrarily choosing a cache, a client chooses the cache the range of the hash function to include the new server, 

server allocated to the document requested. For this 1Q Changing the range of the hash function may result in many 

approach to be successful, it cannot be unduly burdensome documents changing location. For example, if there are (n) 

on the client to determine which cache server to direct a servers and the hash function mapping documents to servers 

document request. It is also necessary to balance the load is f(d)=((ad+b) mod n), then when a server is added, the hash 

appropriately among the caches. For example, even if all function is changed to f(d)=((ad+b) mod (n+1)). 

documents are accessed by clients at the same rate, if some M e le of the ch of m of documents l0 

caches contain more documents than others those caches serveK caused b a ch jn ^ numbef of &ervers {& 

wiU have more requests to service, and may become over- shown ifl nG 3A ^ p f G 3R ^ ^ lhe 

y . . r t.- ul , * , . assignment of 10 documents to 4 servers using the hash 

In the context of caching, it would be unduly burdensome f/ , A t v , A id •n„,,„, fl c ?i, fl 

... i- . # 11 a. j . -i ui f - .u u function ftdWd+l) mod 4. FIG. 3B illustrates the assign- 
to maintain a list of all the documents available in the cache _ . ^ w /,... , , A . ^ 
servers. Such a list would have to be maintained as docu- 20 nient after one addiUonal server is added and the hash 
ments were added and deleted from the original site. An ^ nctl0 ° 15 chan ? d 10 **H d+1 ) ™* «• ^ ™S. ^ circles 
alternative approach is for clients to use a standard hash show * e ma PP^S wilh 4 servers and squares show the 
function to determine from which server to request a docu- mapping with 5 servers. Almost every document is mapped 
ment. In FIG. IB, a standard hash function (h) is used by the 10 a different server as a result of the addition of the new 
clients to choose among three cache servers. Client One 110 25 server - 

requests Doc 1 from Server A 100 since h(Docl)=Server A. A characteristic of this hash function and many other 

Client Two HI requests Doc2 from Server B 121 since standard hash functions is that when the range of the 

h(Doc2)=Server B. In this example, Server B 121 does not function is modified, the mapping is completely reshuffled, 

have the requested document, so it requests the document That is, most documents are mapped to different servers than 

from the original site for Doc2 104. Client Three 112 30 they were mapped to before the change. A complete reshuf- 

requests Doc3 from Server C 122 since h(Doc3)-Server C. fling of the distribution of documents to servers is quite 

Server C 122 does not have Doc3, so Server C 122 requests disruptive to a caching scheme. After each server is added, 

the document from Doc3's Original Site 104. In this every server will need to request copies of the documents for 

example, after obtaining a copy of the requested document, which it is now responsible from the original document sites. 

Server C 122 stores the document. When Client Four 140 35 Adding or removing a server, therefore, places a load on the 

requests Doc3, it also requests it from Server C 122, since cache servers and the original sites. 

h(Doc3)=Server C. Server C 122 already has Doc3 since it In addition to the problem of reshuffling data upon a 

previously obtained a copy to service the request from Client change in the number of servers there is another complica- 

Three 112. Server C 122 is able to respond to the request tion. When a change in the number of servers occurs, users 

without placing any additional load on the network between 40 have to be notified of the change so that they can start to use 

Server A 100 and Server C 122 and without requiring Server a new hash function that includes the new number of cache 

A 100 to take on the load of the request from Client Four servers. If users can be notified of the change simultaneously 

140. The hash function serves as a method for users to agree this does not present a problem, but in a network such as the 

on which server is responsible for a document. Internet, such global synchronous notification is very difiB- 

Hashing with a standard hash function has the same effect 45 cult to achieve. It is feasible, however, to assume that 

as distributing the items randomly, because a standard hash notification about server changes can be done by slowly 

function will tend to distribute items approximately evenly propagating the information from system to system, for 

throughout its output range. This is very desirable because a example in a manner similar to the way that network news 

random assignment of documents to servers will spread the is distributed. As a consequence, the set of cache servers 

load of serving the active set of documents among the 50 recognized by each user may differ widely. A user's "view" 

servers. An example of a standard hash function that can be is a term for the set of servers that the user knows exist, 

used to distribute documents to servers is f(d)=((ad+b) mod Because synchronous notification is impossible, different 

n), where document names (d) are integers, there are (n) users are very likely to have different views, 

servers {0, 1, 2, ... , n-1}, and (a) and (b) are some fixed The example of FIG. 3A and FIG. 3B can also be used to 

choice of integers. For most choices of (a) and (b) the 55 illustrate the effects of multiple views of the servers on the 

function distributes documents roughly evenly over the caching scheme. If some users are aware of the addition of 

servers. server 4 and some of the users are not aware of the addition 

Referring to FIG. 2A, an original distribution of one of server 4, then there are two views of the system held by 

hundred thirty-four documents to servers is shown. Here, the users. One view (the "old" view) is {0,1 ,2,3} and the other 

document names are integers, and there are 13 servers {1, 60 view (the "new" view) is {0,1,2,3,4}. Users not aware of the 

2, . . . , 13}. Some servers store many more documents than addition of server 4 use the old mapping of documents to 

others, and thus in the model of equal access frequency these servers represented by the circles. Users aware of the new 

servers are more heavily loaded. FIG. 2B shows the distri- server 4 use the new mapping represented by the squares, 

bution of documents to servers using a standard hash func- which is almost a complete reshuffling of the old mappings, 

tion such as f(d)-((ad+b) mod 13). In the distribution of FI G. 65 To work in a scheme where users have different views, cache 

2B, no server is responsible for a disproportionately large servers must serve requests for the documents allocated in 

share of documents. all users' views. As a result, the number of documents that 
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a server is responsible for, in this example in which one to the next cache in the tree. If the next cache does not have 

server was added, is almost doubled. For example, users the document, the request is forwarded from that cache to 

with the old view use server 2 for the documents 5 and 29, the next cache and so on. If a document is not stored by any 

while users with the new view use server 2 for documents 11 intermediate cache in the tree, the request eventually reaches 

and 36. Server 2 will receive requests for all of these 5 the root of the tree and is forwarded to the original site of the 

documents, thus effectively doubling the requirements on document. In this approach, a cache retains a copy of any 

the server. The total number of documents that a server is document it obtains for a period of time. An advantage of a 

responsible for is called its load. Referring to FIG. 3B, the cache tree is that a cache receives document requests only 

load of server 2 is 4 since there are two documents from each from the caches that come before it, and this limits the 

view. Using a standard hash function, each view increases 10 number of requests that arrive simultaneously. Thus, many 

the load. requests for a document in a short period of time will only 

A further complication results from having a single docu- causc onc rC( 3 ucst 10 thc ori S inal xr ™ of thc document, and 

ment assigned to different cache servers in different views. wJ1 not rcsult in cachc SMVer overload. 
If there are many cheats with many different views, then a 0ne theoretical disadvantage of this system is that the root 

large number of cache servers can be responsible for a 15 receives at least one request for every distinct document 

document. This in turn implies that thc original site is more requested of the entire cache tree. This can overload the root 

likely to become overloaded with requests for copies from with requests if the number of distinct document requests 

multiple cache servers. For example, referring to FIG. 3A S rows t0 ° lar Se, meaning that this approach suffers from 

and FIG. 3B, document 7 is assigned to server 0 in the view potential scaling problems. 
{0,1,2,3} and to server 3 in the view {0,1,2,3,4}. The 20 SUMMARY OF THE INVENTION 

original site for document 7 has to supply a copy to both ^ ^ • 

a j a tl * * i u / 4 u- u One aspect of the invention is a caching system that 

server 0 and server 3. The total number of servers to which r , . , ,. r , 

. . . • j • ii j *i_ j p .i j t prevents server and network overloading and scales grace - 

a document is assigned is called the spread ol the document. 5„ , .... , . , . -. 

v i .1 j c j\ * c • *u i c fully even when clients have multiple views. Resources can 

For example, the spread of document 5 in the example of . J . , , , , - , * i 

n i a j cfr- in * t tt • « be~added-and - removed -from- a ^pooLof ^available resources 

Hu. 3A and MG. 3d is 2. Using a standard hash function, \ . A . A ,-^r=^„s ■ = i .-===r?w,-i7 c~5—--<r- ... ^ — j 

, f . t . **u *u l c without a reallocation orall or even-most of the requests. 

the spread of a document increases with the number of ~ . . , .,_ . , , ,j , j 

2 n iYJ3P e requ ests^ associa ted^ith~the seryer _added ^or 

removelJ^hav^to^be-reanocatedrrThis is referred tolas 

In summary, the use of standard hash functions to identify "cxmsistenthaslimg^ requests? 

a resource (for example, a cache server) for a particular fthTldlocafio^is^^ added and? 

request (for example, a document) results in a greatly removed.-Onlyl^b^^ 

increased load and spread when the number of resources different resources. With regard to caching data, consistent 

(cache servers) changes, and there are, accordingly, multiple hashing avoids increasing load ^ sp read wnen me number 

views. This drawback can have a great impact on perfor- 0 f servers changes 

mance when the number of documents and servers is large. t „ nc . n n£ „ . w , ♦ 

& 35 In anotner aspect, a series ox servers is used that are 

It should be noted that the approach of allocating respon- chosen in a manner that spreads the load associated with a 

sibility for each document to different servers is less effec- sma n number of frequently requested documents, and 

tive when the cache is experiencing a large number of thereby prevents overloading. This is referred to as 'Sdrtual 

requests for the same small number of documents. The large networks." When used together, consistent hashing and 

number of requests will all go to the small number of servers ^ virtual networks improve the ability of document servers 

responsible for the small number of documents The small generally and caching servers in particular to respond to 

number of servers are likely to be overloaded by the large j ar g C numbers of requests. 

number of requests. In one aspect9 me invention relates to a method and 
Onc approach for preventing one server from being apparatus for distributing a request to one of a plurality of 
overloaded by a large number of requests for the same 45 resources that includes mapping the request to a request 
document is to use a group of caches in scries between the location in mathematical mapping space, mapping each of 
user and the original site of the document, each of which can the resources to a respective at least one resource location in 
service thc request for thc document. In one embodiment of mathematical mapping space, and allocating the request to 
this approach, if a first cache server does not have a one of the resources based on a mathematical relationship 
document, it requests the document from another (second) 50 between the request and resource locations. An apparatus for 
cache and not the original site. The second cache services a distributing a request to one of a plurality of resources 
defined group of first-level caches. If the second cachc does includes the plurality of resources and an input receiving a 
not have the document, the second cache requests the resource request. A mapper in communication with the input 
document from yet another (third) cache. The third cache maps the request to a request location in a mathematical 
services a defined group of second-level caches. This con- 55 mapping space and maps each of the plurality of resources 
tinues until the last cache in the series does not have the to at least one resource locations in the mathematical map- 
document, at which point the last cachc requests the docu- ping space. A distributor distributes the request to one of the 
ment from the original site. This approach provides cache plurality of resources in response to a mathematical rela- 
layers between the first cache and the original site and it tionship between the locations of the request and resources 
spreads the load among the caches at each layer. Different 60 in the mathematical mapping space, 
users can use different lists of caches for the same document, i D one embodiment, the mathematical mapping space is a 
thereby spreading the load. The list of servers to request the c [ TC \ Gt \ Q one embodiment the request is a data request and 
document from can be derived in various ways. the resource is a server. In one embodiment the step of 
One approach is to use a tree of caches. In this approach, mapping each of the resources to a respective at least one 
a user requests a document by requesting the document from 65 resource location comprises hashing each of the resources to 
a server representing a first node on a tree of caches. If this a respective resource location in the mathematical mapping 
cache does not have the document, the request is forwarded space. 
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The invention also relates to a computer-readable medium the second data request. In one embodiment the second data 

whose contents cause a computer system to distribute a request includes a sequence of nodes. In one embodiment 

request to one of a plurality of resources. The computer the method further includes receiving the data from the 

system executes the contents of the computer-readable server mapped to the next node in response to the second 

medium by performing a computer program. The computer 5 data request, determining the total number of times the data 

program maps the request to a request location in a math- in the first data request has been requested, and caching the 

ematical mapping space and maps each of the resources to data requested by the first data request if the count of the 

a respective at least one location in the mathematical map- number of times the data in the first data request was 

ping space. The computer program allocates the request to requested exceeds a threshold. In one embodiment the 

one of the resources based on a relationship between the 10 method includes the step of responding to the first request by 

request location and the resource locations in the mathemati- transmitting the data. 

cal mapping space. The invention also relates to an apparatus for responding 

Another aspect of the invention relates to a method for to data requests made by a client. The apparatus includes an 

requesting data from one of a plurality of servers. The input receiving a first data request that includes a sequence 

method includes generating an virtual network of nodes in 15 of nodes corresponding to a sequence of servers. The 

response to a data request and randomly choosing a first apparatus also includes decision logic determining whether 

node on the virtual network of nodes. The method also the data specified in the data request is available on a 

includes generating a path from the first node to the root selected server, and if the data is not available determining 

node on the virtual network of nodes and mapping at least a next node from the sequence of nodes. The apparatus also 

one node on the path to a respective one of the plurality of 20 includes a mapper mapping a server to the next node. The 

servers. The method also includes requesting data from the apparatus also includes an output requesting the data by 

one of the plurality of servers. transmitting a second data request to the server mapped to 

In one embodiment, the step of requesting data from one the ncxt node, 

of the plurality of servers is accomplished with a data BRIEF DESCRIPTION OF THE DRAWINGS 
request comprising identification of the desired data and a 

sequence of nodes through which the request should be FIG * 1A * a block diagram of clients requesting docu- 

directed. In one embodiment, the data request also com- ments from a cacne xt}/eT ^ known to the prior art; 

prises a sequence of servers that should act as the nodes. In FIG. IB is a block diagram of clients using a hash 

one embodiment requesting the data from the server mapped function to request documents from different cache servers 

to the chosen node includes transmitting a request from a 30 as known to the prior art; 

client to the server. In one embodiment mapping at least one FIG. 2 A is a graph of the distribution of documents to 

node on the path to a respective one of the plurality of servers as known to the prior art; 

servers comprises calculating a hash function. In one FIG. 2B is distribution of documents to servers using a 

embodiment mapping a node on the path to a respective one standard hash function as known to the prior art; 

of the servers includes mapping the node to a node location 3 FIGS 3A and 3B m distributions of documents to four 

and each of the servers to at least one server location in a and five servers using a standard hash function as known to 

mathematical mapping space and allocating the node to one me pr j or art . 

of the servers based on a mathematical relationship between a a flrt „ u . , ... . c , c 

t . , . « , • , FIG. 4A is a flowchart of an embodiment of a method for 

the node location and the server locations m the mathemati- „ . . A . 

. 40 allocating resources constructed according to the present 

cal mapping space. & & r 

vr & * invention; 

The invention also relates to an apparatus for requesting RG. 4B is a block diagram of an embodiment of a system 

data from one of a plurality of servers. The apparatus fof allocating resources ^ oostnscle(l according t0 tne pr esent 

includes an input receiving a data request and a path invention 1 

generator in communication with the input that randomly IC ' . ... , _ . 

. j i *. i I , .i * 45 FIG. 4C is an embodiment of a mathematical mapping 

chooses a node on an virtual network and generates at least - , .... 

t t r> j . «u j . * t'l. space or the present invention; 

one path of nodes in response to the data request. The , « . - 

apparatus also includes a mapper in communication with the FI 9 S * 5A and 5B are embodiments of mathematical 
path generator mapping nodes on the at least one path of ma PP in & s P accs accordin S to the present invention; 
nodes to one of the plurality of servers. The apparatus also - p IG. 5C 1S a fi ra P h of an example of distribution changes 
includes an output in communication with the mapper and mat occur when a server « added t0 a network according to 
requesting data from the one of the plurality of servers an embodiment of the present invention; 
mapped to the nodes on the at least one path of nodes. FIG. 6 is an embodiment of a mathematical mapping 
In another aspect, the invention relates to a method for s P ace in which me P oints are distributed badly in the space; 
responding to data requests made by a client. The method S5 p IGS. 7 A, 7B, and 7C depict an embodiment of a math- 
includes receiving a first data request that includes a ematical mapping space according to the present invention; 
sequence of nodes corresponding to a sequence of servers. FIG. 8 depicts an alternative embodiment of a mathemati- 
The method also includes determining whether the data cal mapping space according to the present invention; 
specified in the data request is not available on a selected FIG. 9 is a block diagram of an embodiment of an 
server. If the data is not available, the method also includes 60 apparatus according to the present invention; 
determining a next node from the sequence of nodes, map- FIG. 10 is an embodiment of a group of servers according 
ping a server to the next node, and requesting the data by to the present invention; 

transmitting a second data request to the server mapped to FIG. 11 is an embodiment of an virtual network according 

the next node. to the present invention; 

In one embodiment the step of mapping a server to the 65 FIGS. 12 and 13 are each mappings of the virtual network 

next node comprises calculating a hash function. In one of FIG. 11 to the group of servers of FIG. 10 according to 

embodiment the first data request is substantially similar to an embodiment of the present invention; 
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FIG. 14 is a palh on the virtual network of FIG. 11 is preferred that allows for efficient mapping and matching 
according to an embodiment of the present invention; of resources to requests. In the embodiment of FIG. 4B, the 
FIG. 15A depicts the change in the mapping of an mathematical mapping space is a two dimensional space of 
embodiment of a virtual network to servers when a node and real numbers. In another embodiment, the mapping space is 
server are added, using a standard hash function; 5 a line. In another embodiment, the mapping space is the 
FIG. 15B depicts the change in the mapping of a virtual three dimensional space of real numbers. In another 
network to servers when a node and server are added, using embodiment, the mapping space has more than three dimen- 
the consistent hashing of an embodiment of the present sions. In another embodiment, the mapping space is a circle, 
invention; and ] D one embodiment, the mapping space is a circle with a 
FIG. 16 is an alternative embodiment of a virtual network 10 circumference of one. In yet another embodiment, the map* 
according to the present invention. p mg sp ace is an ellipse. In another embodiment, the map- 
Like reference characters in the respective drawn figures ping space is a three dimensional sphere, 
indicate corresponding parts. In orjc embodiment, the mapping of requests and 
DESCRIPTION OF THE PREFERRED resources to the mathematical mapping space is accom- 
EMBODIMENT 15 plished with a hash function. A hash function takes an input 

o * . . u u- 00 and returns an output in a given range. Hash functions 

Consistent Hashing w . . . . r . . ^ . & , . 

, - , . . , can be used to allocate pointers or keys to a location in a 

In one aspect of the present invention a mathematical hash ^ whkh ^ a data hayin a ^ 

mapping space is chosen and a mappmg is used to locate numbero f locations or "slots." If h(k) is a hash function, key 

requests and resources m the mathematical space^A specific 20 w h ^ to hash tQ a hash toHe location h(k) A ^ ^ 

mattiemalical relationship .s also chosen to match a request dcsi ^ a hasn ^ for cach k (k) to bc u 

with a resource in the mathematical space. A request is ,„ nash to of ^ ]ocatif)ns m ^ hash ^ ^ 

assigned to the resource whose location in the mapping ^ ^ achicve ^ k to derive ^ hash 6ae&m ^ a 

space has the specific mathematical relationship with the ^ is , ed , Q be md den , of an ttcms that 

request s location in the mathematical mapping space. 2s might cx js t in the data 

Referring to FIG. 4A, a request is mapped to at least one . 

location in the mathematical mapping space (Step 492). u Most n f n factions assume that the universe of keys is 
Available resources are each mapped to at least one location * e » 1 of natural numbets ' J f °» m P ut value 10 * hash 
in the mathematical mapping space (Step 494). These steps, bincll0D 15 » ot a natura ^ DUmber ' w »* 18 "^^ found 
step 492 and step 494, do not have to occur in the order 30 to aterpret the mput value as a natural number. For example, 
shown, each can be accomplished independently from the rf a string is to be used as input to a hash function, 
other. In fact, in one embodiment, the resources are mapped ll be i^rpreted as an integer expressed m suitable radix 
once, and the mapping is stored for future use with each notatlOD - ™ example the string "mit.edu" could be 
request. Having mapped the request and the resources to the interpreted ">l*S« by using the ASCII decimal con- 
mathematical mapping space, the request is allocated to a 35 ve ^°°°f^ letters in the string, and treating each character 
resource based on a mathematical relationship between the « » ^^^^•^^\^ t ^lfT : 

By way of example, in the embodiment of FIG. 4B, a In a division method for creating hash functions, a key (k) 

client wants to request a document from a server. In this 40 fe ma PP ed inl ° 0De of m locations by taking the remainder 

embodiment, the request is a document request and the of 00 divided by (m). That is, the simplest form of such a 

resources are cache servers Server A 400 and Server B 421. hash function looks like h(k)=(k mod m). Using this simple 

The client 410 requests Docl from a server. Referring also example, if the hash table has size (m=12) and the key is 

to FIG. 4C, the client 410 selects Server A 400 by first (k-100), then h(k)=4. Since it requires only a single division 

mapping the request for Docl and the available servers 45 operation, hashing by division is quite fast. More compli- 

Server A 400 and Server B 421 into a previously selected caled hash factions can be constructed using a division 

mathematical mapping space. An embodiment of a math- method, for example, h(k)=((ak+b) mod n) for some fixed 

ematical mapping space 450 is a two dimensional space of choice of integers (a) and (b), and (n) locations, 

real numbers. Docl 408 is mapped into the mathematical In one embodiment, a family of hash functions is used 

mapping space 450, as are Server A 400 and Server B 421. 50 whose elements arc cach functions that map requests and 

In this embodiment, the resource that is the least distance resources to locations in mathematical mapping space. A 

from the request is allocated to the request In other random function (f) is chosen from such a family of hash 

embodiments, other mathematical relationships could be functions (H) and function (f) is used to map requests and 

used. Client One 410 then allocates the request to Server A resources to locations. This family model of hashing is used 

400 based on the distance relationship between Docl and 55 because schemes that use a single fixed function are vul- 

Server A 400 in the mathematical mapping space. The nerablc to attack by parties attempting to damage the system, 

distance 418 between Docl 408 and Server A 400 is less since the input that causes the absolute worst case behavior 

than the distance between Docl 408 and Server B 421. Docl of the scheme can be determined. 

408 is therefore allocated to Server A 400. The client 410 For example, if an adversary has the power to selectively 

then requests Docl from Server A 400. likewise, the client 60 disable some number of resources, and the mapping of 

410 requests Doc2 from Server B 421 after mapping Doc2 requests and resources to the mapping space is known to the 

and the servers into mathematical mapping space 450 and adversary, then the adversary can selectively disable those 

determining that the distance 420 between Doc2 410 and resources that would leave a remaining resource heavily 

Server A 400 is more than the distance 422 between Doc2 loaded, and possibly overloaded. The adversary can elimi- 

410 and Server B 421. 65 na t e all resources in a portion of the mapping-spacercreating^ 

The mapping space may be any mapping space in which a gap witrTno resources at all. All:me:requests^previously 

the requests and resources can be mapped. A mapping space asslE^dJo.Uie-elim^ 
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< toTrcsource whose location is on the edge of the gap. This traversed either clockwise or counter-clockwise from the 

resource might- then become overloaded "with requests^A request's location. In other embodiments, the request may be 

way to foil such an intentional destruction of the system is assigned to the resource whose location is any specific 

to add randomization to the scheme. For example, if the number of points from the request location, 

locations in mapping space associated with requests and s Qnce a { bas bwn matched wi , h a KSOUKe accQrd . 

resources are chosen randomly and an adversary is no . ^ re ? a(ionshi of ^ „ , ^ the fesource in the 

aware of the random choices, then the adversary does not & ( , t . . " 2 * ■ n . j * , L 

know the locations of the requests and resources in the mathematical mapping space, the request is aUocated to the 

mapping space. Therefore the adversary does not have the * sourcc - la 0DC embodiment, the allocation is reported to 

necessary information to selectively disable resources lft * c operating system user, browser ,or other entity that will 

located in a particular part of the mapping space. Another 10 ^formation. In another embodiment, the allocation 

technique to foil such an attack is to map multiple points for P lat f automatically so that the reporting actually 

each resource into the mapping space. This is discussed allocates the resource. 

further below. 1° one embodiment shown in FIG. 5 A and FIG. 5B, 

Randomization is added by choosing a mapping function 15 documents are allocated to a plurality of caching servers, 

from a family of hash functions so that there is a uniform ^ mathematical mapping space that is used is a circle that 

probability of choosing every function. Thus, using a hash has a radius of Both documents and servers are mapped 

family (H) means that first a function (feH) is selected and to P oints 0D circle a hash function, as indicated by 

then that function is used to map requests to resources. For document points D1-D8 and server points S1-S6 on the 

example, if both the set of (request and resource) items (I) M circle. A document is to be assigned to the closest server 

and the set of (location) buckets B are in the range {0,1, movin S clockwise around the circle. Arrow CW shows the 

2, . . . , p-1} for a prime p, then the family (H) could be clockwise direction of mapping of documents to servers. In 

denned as all the functions of the form f(x)«((ax+b) mod p) ^is embodiment, Document Dl is assigned to server SI, 

for all a, b e {0,1,2, . . . ,p-l}. To use this hash family, a and document D2 is assigned to server S3, documents D3 and 

b are chosen at random, and this is equivalent to choosing a ^ 04 are assigned to server S4, document D5 is assigned to 

function from the family at random. Then, a request or server S5 » ^ documents D6, Dl, and D8 are mapped to 

resource item is assigned to location ((aR+b) mod p). In server S6. 

other words, the probability that a randomly chosen function FIG. 5B depicts an embodiment of the same system after 

(f) from the family (H) maps the request (x) to resource (i) a new server is added. The new server is shown as Server S7. 

and the request (y) to resource (j) is the same as in the case 30 With the addition of the new server, the only documents that 

in which the two requests are mapped to the resources are reassigned are those now closest to the new server going 

uniformly and independently. The family (H) is used instead clockwise around the circle. Thus only documents D6 and 

of choosing a completely random function because D7 move to Server S7, other documents remain with the 

choosing, storing, and evaluating functions from H is simple server they were previously matched with. FIG. 5C shows 

and efficient. Constants (a) and (b) can be chosen at random. 35 the change in the mapping of documents to servers before 

These values can be stored using very little memory, and the and after the addition of a new server. Circles are the 

resulting function can be evaluated simply. On the other previous mapping (as in FIG. 5A) and squares show the new 

hand, selecting a truly random function requires choosing mapping (as in FIG. 5B). This shows a significant improve- 

and storing a random table containing the value of the ment over the results in FIG. 3 in that many fewer docu- 

function for every single item or seeding a known random ^ ments have changed servers with the addition of a new 

number generator with a known value. Such a hash function server. In fact, only the documents associated with the new 

may be implemented in hardware or software. Server S7 are reallocated. 

Although randomization helps balance load, the mapping In this embodiment, if a document is assigned to a server 

does not have to be entirely random. Other information in one view, then that document's location in the mapping 

about the request of the resource can be used to map the 45 space is likely to be relatively close to the server's location 

request or resource into the mathematical mapping space. in the mapping space in all views. This is because another 

For example, geographic information may be useful for server would probably be in between the document and the 

placing requests and resources located geographically near server if the document and the server were not close to each 

each other at least somewhat proximate in the mathematical other. If, in this embodiment, server points arc evenly 

mapping space. Similarly, it might be helpful for the location 50 distributed around the circle, only a few document points fall 

of a request or resource in a network topology to influence close to a server's point. Even if there arc a large number of 

the location in the mathematical mapping space. views, only a relatively small number of documents will be 

For example, if the mathematical mapping space is a grid assigned to a server, and the load should not increase more 

with geographical map coordinates, the mapping function than logarithmically with the number of views. Likewise, 

could be a random distribution around the geographic loca- 55 even if there arc many views, only a few servers will have 

tion of the resource. The random distribution would help to responsibility for any one document. The spread of a docu- 

balance the load, and the geographic bias would help match ment therefore does not increase dramatically with the 

requests and resources that are geographically proximate. number of views. These properties allow a caching scheme 

The mathematical relationship that is used to match to grow gracefully. Servers can be added to the network 

requests with resources can be any mathematical relation- 60 without disrupting the caching and multiple views of the 

ship that allows requests to be matched with resources. A servers can exist without degrading the performance of the 

mathematical relationship is preferred that can be deter- system. 

mined efficiently. In the embodiment of FIG. 4C the resource The benefits of decreased load and spread depend on an 

that is the least distance from the request is allocated to the even distribution of the resources in the mapping space. In 

request In other embodiments in which the mathematical 65 one embodiment, requests and resources are mapped to 

space is a circle, a request is assigned to the resource whose random points around a circle, and a request is mapped to the 

location is the first encountered when the mapping space is resource whose point is encountered first when the circle is 
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traversed clockwise from the item's point. In this resource clockwise around the circle. In this way, a path is 

embodiment, there is a small probability that points could be determined of resources to query for a request. In one 

distributed badly around the circle so that one resource is embodiment, this is combined with the virtual networks 

responsible for a disproportionate section of the circle. Such discussed below, but it can also be used instead of the virtual 

an unfortunate instance is shown in FIG. 6, in which points 5 networks. 

6A-6F are all on one side of the circle. All request points on In another embodiment, a request is always mapped to the 

the left half of the circle would be mapped to resource 6A. sai *i e location in the mathematical mapping space. Each 

In one embodiment, in order to decrease the likelihood of resource is ma PP' d mto ll ! e ^thematical mapping space 

, , , ' j * . *» t ■ mr / using a mapping function that is a function of the resource 

such an unlucky event as the poor distribution of FIG. 6, a and ^ £ st s Because lhe resource { ft|nction ^ a 

resource is assigned to two or more locaUons in the mapping «> h ^ ^ ^ ^ 

space instead of just one. In this embodiment requests are of mc rcsourccs m mc mapping spacc dcpcnd> m part> 

still assigned to a single point on the circle and are mapped on thc rcqucst Refcrring to mGm 8> m embodiment of a 

to the resource that has a point nearest to the request's point circular mathematical mapping space is shown where the 

in a predetermined direction. When the number of resource request 801 is always mapped to the origin point at the top 

points (m) is larger, there is less of a chance that the points 15 0 f the circle. The resources 802-309 are mapped to the 

will be unequally distributed around the circle. The number mathematical mapping space using a mapping function that 

of resource points (m) need not be unmanageably large for is a function of the resource and the request. Such a function, 

there to be a very good chance of an even distribution of for example, might be (((f(i)*ax)+b) mod n), where f(i) is a 

points in the mapping space. function of the request (i), (x) is the resource to be mapped, 

In one embodiment, the mapping space is a circle as in 20 ( a ) and 00 m constants, and (n) is the number of points 

FIG. 7A. In this embodiment, the mapping function is around the circle. The result is that for one value of (i), (for 

chosen from the family of standard hash functions y-(ax + b) ex ¥!P te ' £jO> resource < x > ™ U „ be ma PP e * to ™ e l*»*>n, 

mod n. Hie number of points (n) is the number of possible for a ^rent value of (i), (for example, i-2), resource 

* j u j* ( x ) ma Y De mapped to a different location, depending on ft l), 

points that the requests and resources can be mapped to. a/* ■ j j- l. «l csa\ ■ j-a * <i? r/V 

u c ? » / \ j i_ j i . 25 fo at is> depending on whether r(l) is different than f(2). 

Four sets of constants (a) and (b) are chosen random y to Refc ^g t ^ F i G . ^ me step of mappmg the request ks\ep 

map the resources and one set of constants (a) and (b) are 492 wouM map the request {Q me same location 

chosen to map the requests. Mapping the resources into the mathematical mapping space 

The use of multiple points has the added benefit of making in step 494 would occur using a mathematical mapping 

it much harder for an adversary to create holes in the 3Q function that takes into account both the request and the 

mapping by selectively eliminating a small subset of the resource. 

servers, thereby overloading a remaining server and causing Referring to FIG. 9, an embodiment of an apparatus 

it to crash, which will then overload another server and according to the present invention includes an input 810, 

cause it to crash, and so on. If the multiple points for each which receives a resource request 815. The input 810 

server are evenly and randomly distributed in the mapping 3$ receives the request 815 and passes the request 815 on to the 

space, the elimination of one server will not put a great distributor 830. In one embodiment, the input 810 receives 

burden on any one or subset of the remaining servers. a request 815 over a computer network. In another 

Referring to FIG. 7A, resources Bl and B2 each have 4 embodiment, the input 810 receives a request 815 from 

points on the circle. Requests are mapped to the resource another process on the same computer. The input 810 can be 

closest to the resource in the mapping space as measured in 40 implemented in hardware or software, 

a clockwise direction. Request 701 is closest in a clockwise The distributor 830 uses the mapper 820 to map the 

direction to a point of resource Bl and request 702 is closest request 815 and the resources 850 into a mathematical 

in a clockwise direction to another point of resource B2. mapping space and to determine the allocation of the request 

Requests 703 and 704 are closest in a clockwise direction to 815 to one of the resources 850. In one embodiment, the 

another point of resource Bl. Request 705 is closest in a 45 mapper 820 includes mapping space data stored in a binary 

clockwise direction to another point of resource Bl. In FIG. tree structure for fast access. Each resource is stored in the 

7B, the circular mapping space is drawn as a linear interval binary tree, and then the request 815 can be matched to one 

of length one where we imagine that the endpoints of the 0 f the resources 850 by searching the tree. Other types of 

interval are "glued together." The points on the line mark the storage structures can be used as well, for example an array 

intervals shown in FIG. 7C. The intervals indicate the 50 or linked list. 

mapping space that resources Bl and B2 are responsible for. i D one embodiment, the mapping space store has a func- 

Resource points are responsible for the interval from the tion AddResource(r), which adds a resource to the store. It 

location of the resource to the next point directly to the left. a js 0 nas a function DeleteResource(r), which deleted a 

Since there are multiple copies of each resource, resources resource from the store. It also has a function MatchRequest 

are responsible for a set of intervals. 55 ^ whicn pro vides the resource that is greater than or equal 

When a resource is mapped to multiple points, there is a to (r). If there is no such resource, meaning that the request 

small but positive probability that when a random function has wrapped around the circle, the MatchRequest(r) function 

is chosen from the family, two points from one resource provides the resource at the smallest numbered point. The 

could be mapped to the same location in the mapping space. distributor 830 accesses the mapper through the functions 

In one embodiment, if this happens, some arbitrary fixed 60 just described. By using the MatchRequest(r) function, the 

order of the assigning the points on the circle is used. In distributor 830 can allocate the request 815 to one of the 

another embodiment, the probability that such a function is resources. The mapper 820 can be implemented in hardware 

chosen is made arbitrarily small by using more bits of or software. The mapper 820 can store the locations of the 

precision. resources in a tree structure as described, or it can receive a 

In one embodiment, a request is made to the first resource 65 list of the resources 850 to be mapped with each request 815. 

in a particular direction around the circle. If that resource In one embodiment, the distributor 830 periodically 

does not have the item, the request is made to the next obtains information about the resources 850 that are 
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available, so that it can use the AddResourceO and of FIG. 11 to the servers of FIG. 10. In this mapping, root 

DeleteResourceQ functions to keep the view current. After node 1101 is mapped to Server One. Middle Node A 1102 is 

selecting one of the resources 850 to allocate to the request mapped to Server Two. First Node A 1105 is mapped to 

815, the distributor 830 allocates the request 815 to the one Server Five. FIG. 13 shows an alternate mapping of the 

of the resources 850, and outputs any information necessary 5 virtual network of FIG. U to the servers of FIG. 10. In this 

to allocate the request 815 to the resource via output 840. mapping, root node 1151 is mapped to Server Thirteen. 

Virtual Networks Middle Node A 1152 is mapped to Server Ten. First Node A 

The consistent hashing method and apparatus just is mapped to Server One. 

described is particularly useful when combined with another These are just two of the many possible mappings of the 

aspect of the present invention, in which a virtual network 10 virtual network of FIG. 11 to the servers of FIG. 10. The 

of cache servers is used to prevent any server from having examples of FIG. 12 and FIG. 13 demonstrate that the 

too many requesters asking it for a particular document. The different mappings from virtual network to servers map 

virtual network does not require an unmanageable number different servers at each level of the network. For example, 

of messages to be communicated among the servers, and at in the mapping of FIG. 12, Server One is mapped to the root 

the same time provides for a low probability that any server 15 node 1101. In the mapping of FIG. 13, Server One is mapped 

will be overloaded. A different network of servers is used for to First Node A 1155. If different mappings are used for 

each particular document. Use of a different network for different documents, there will be some documents for 

each document spreads the load so that none of the servers which a particular server will be the root, and there will be 

is at the same part of the network for a disproportionate other documents for which that server will be a first node or 

number of documents. 20 middle node. This distributes the load associated with a 

In one embodiment, a client requesting data from a group particular position in the network, 

of servers uses a virtual network of nodes that are not Mapping of the virtual nodes to servers can be accom- 

initially mapped to servers. The client chooses a first node on plished using a standard hash function. The mapping can 

the virtual network to be the node that will receive the also be accomplished with the consistent hashing of the 

client's request. The root node is the last node before a present invention. Consistent hashing provides the added 

request is passed on to the original site for the document- benefit of allowing servers to be added and removed without 

The choice of a first node may dictate, at least in part, a path disrupting the caching scheme. 

of nodes from the first node to a root node on the virtual In one embodiment, none of the virtual nodes 1001-1013 

network. As described further below, the client then maps 3Q are initially mapped to a particular server. A first node on the 

the nodes on the path of nodes in the virtual network to virtual network is chosen. In one embodiment, the first node 

actual servers. Having mapped the nodes to servers, the is chosen randomly from all of the nodes on the virtual 

client requests the data from the server mapped to the first network. In another embodiment, the first node is chosen 

node. In one embodiment, the request includes the path of from a subset of the first nodes determined by the identity of 

nodes and the servers mapped to those nodes. If the server 35 the client. In another embodiment, the first node is chosen 

mapped to the first node does not have the document, the * from a subset of the first nodes determined by the document 

server mapped to the first node requests the document from requested. The choice of a first node dictates, at least in part, 

the server mapped to the next mode on the path from the first a path of nodes from the first node to the root node. For 

node to the root node in the virtual network of nodes. If this example, FIG. 14 shows the virtual node network of FIG. 11 

server does not have the document, it will in turn ask the 4Q in which first node D 1208 on the virtual network has been 

server corresponding to the next node on the path. Each chosen. This dictates a path from first node D 1208 to next 

server that maps to a node on the path from the first node to node B 1203 to root 1201. 

root node may ultimately be queried for the document if the Once a first node has been chosen and a path of nodes 

document is not located on the proceeding node. Finally, if determined, the nodes on the path of nodes can be mapped 

the server that maps to the root node does not have the 45 to the actual servers. Each node on the path can be mapped 

document, the server that maps to the root node will request to a server. In a preferred embodiment, the mapping of nodes 

the document from the original site for the document. to servers is based on the name of the document requested. 

Referring to FIG. 10, a group of thirteen cache servers Having mapped the nodes to servers, the client requests the 

901-913 arc individual computers, each with their own data from the server mapped to the first node. In one 

RAM, ROM, processors), and storage media. The servers 5 0 embodiment, the request includes the path of nodes and the 

901-913 are connected by a network 916. Together, servers servers mapped to those nodes. If the server mapped to the 

901-913 handle requests for documents. The number of first node does not have the document, it will request the 

servers is not central to the invention. There can be as few document from the server mapped to the next node of the 

as two servers, and an unlimited maximum number of first node in the virtual network. 

servers. The type of server is also not central to the inven- 55 In another embodiment, each node on the virtual network 

tion. In another embodiment, the servers are each individual of nodes is first mapped to a server, thereby determining a 

processor boards connected to a single bus and network network of servers for a particular document. A first server 

interface. In yet another embodiment, the servers arc indi- on the network of servers, and a path from the first server to 

vidual software processes that execute on a multithreaded the root can then be chosen for a request. While this ordering 

computer system. 60 of steps is also a functional embodiment of the invention, it 

An embodiment of an virtual network of nodes that can be has the disadvantage of requiring all nodes to be mapped, 

used with the server group of FIG. 10 is shown in FIG. 11. Choosing a path on the virtual network first, and then 

This particular embodiment shows a tree structure, but the mapping the nodes on the path second, requires fewer nodes 

invention is not limited to tree networks. Each of the thirteen to be mapped to servers, and therefore requires less calcu- 

nodes 1001-1013 represents one cache. Each node 65 lauon ume - 

1001-1013 can be mapped to a server 901-913. For In another embodiment, the request does not include the 

example, FIG. 12 shows one mapping of the virtual network path of nodes and servers mapped to those nodes. In this 
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embodiment, each server that needs to request the document 
from another server uses the information about the docu- 
ment requested to create its own virtual network, map nodes 
to servers and determine the next server. In this embodiment, 
the client need only determine the server mapped to the first 5 
node, and not the entire path. 

^Jn-one-embodirnent, when a server receives-a~requestj it 
first checks to r see if it is caching -a : copy~of the documentor 
isan-the.process of getting oneto cacherlf soptTefufn , l the^ 

'document to the requeste r^ter-itgets its copy-if necessary. 

|Othej^^~jrincTemejns a request counter for the document 
requesle^ and_me~virtual "no^OJ 

nexrsefver on the path for the document? If the request 
counterreaches:ardefined:number of requests, when acting-^ 
as a particular virtual node,Jhen ^the^cache_seryer^ cach~es - a 
copy of the docurnenl. In either^case-me cache passes;the 1 
document on to, the requester ' when it is. obtained. / 

When consistent" has^ngis used~to~map nodesto servers, 
there is an additional benefit that servers can be added and 
removed without disrupting the caching scheme. Referring 
to FIG. 15 A, an embodiment of a virtual network is shown 
in which two clients CI and C2 have different views. CI is 
aware of servers {1 ... 6} and C2 is aware of servers {1 . . . 
7}. FIG. 15A shows a virtual network mapped to servers 
using a standard hash function for a particular document for 
the two views. In the first view of FIG. 15A, CI starts with 
the server mapped to node 5. The path from first node to root 
is 5-1-3. In the second view, the servers are mapped very 
differently because of the addition of a new server. C2 starts 
with the node mapped to server 6, and the path is 6-7-2. No 
servers are the same. 

Referring to FIG. 15B, the same views are shown, but 
with the mapping of nodes to servers using consistent 
hashing. In the first view, CI first requests from the node 
mapped to server 6. The path is 6-4-1. In the second view, 
using consistent bashing, the mapping is much closer to that 
of the first view than it was with the standard hash function. 
The first node is the node mapped to server 7. The path is 
7-4-1. Since the nodes mapped to servers 4 and 1 are the 
same servers in both views, it is likely that server 4 or server 
1 will have a copy of the document. 

Referring to FIG. 16, the disclosed embodiment of a 
non-hierarchical virtual network is a circle with spokes 
connected to center node 16F. The root of this virtual 
network is the center node 16F, and any node 16A-16E on 
the circle can be the first node. As described above, a 
hierarchical network has beneficial properties, such as being 
useful for reducing hot spots. Other virtual networks, like 
the one in FIG. 16 have other benefits, such as a resemblance 
to an existing network topology, or utility for a particular 
routing protocol. As in the hierarchical network, a first node 
is chosen to receive a particular request, and a path chosen 
from the first node to the root node for that request. For the 
embodiment of FIG. 16, the path is chosen by a clockwise 
traversal of the circle. When the requests have gone around 
the circle and reached the node before the first node, the 
request is then passed to the root. After the path has been 
chosen, the nodes are mapped to servers. The nodes can be 
mapped to servers using a hash function that takes into 
account the information being retrieved. The hash function 
may also lake into account other factors. Use of the consis- 
tent hashing described above provides the added benefits 
discussed earlier, such as the ability to add and remove 
servers without disrupting the caching scheme. 

Having described preferred embodiments of the 
invention, it will now become apparent to one of skill in the 
art that other embodiments incorporating the concepts may 
be used. 



The invention as described with regard to cache servers 
works equally well for general storage of objects in a 
computer or computer network, where the requests arc data 
to be stored or retrieved from a data store and the resources 
are the data store. For example, the invention is applicable 
to a collection of documents that spans multiple disks, or 
other large data store, within a single computer system. 
Consistent hashing can be used to allocate documents to 
disks. This would help balance load, and allow for addition 
and removal of disks with a minimum data reorganization. 
Virtual networks of caches could be used in such a system 
between the disks and one or more processors. This would 
particularly be useful for a computer system storing many 
documents on many devices. 

Networked distributed file systems, groups of computers 
that store large amounts of data on an intranet or internet, 
and have too much information to effectively serve on one 
server, can use the invention to great benefit. Consistent 
hashing can be used to allocate documents to servers, 
thereby making it possible for servers to be added and 
removed from such a system with minimal impact on the 
locations of much of the data stored in the servers. A virtual 
network of caches can stand between the users of the 
distributed file system and the distributed file system, to 
spread the load on the servers. 

An example of such a system is a large library of 
documents. Another example is a personal file service. Such 
a service would allow a user to store his files on a collection 
of central servers. Any device connected to the network 
having proper authentication could have access to those 
files. The files could include such computer data such as 
electronic, video, and voice messages, commonly referred to 
as email and voicemail. Such a personal file service, if 
implemented for large number of users, would benefit from 
the consistent hashing and replication of the present inven- 
tion. It could, for example, enable large scale military 
command and control systems. Such a system could deliver 
messages that simulate a broadcast, by allowing many users 
to access files at the same time. 

Another application of the present invention is in the area 
40 of servicing requests for suddenly popular web pages. A 
server system using the present invention can provide ser- 
vice for "hot" web sites that experience short-term traffic 
surges. For example, sites providing a new release of 
software, recent photographs from outer space exploration 
45 (as occurred with the Mars lander), and sites reporting scores 
during a sports event of limited duration (such as during a 
college basketball tournament) receive extremely high vol- 
umes of requests for relatively short periods of time. The 
short periods might not justify a significant hardware invest - 
50 mcnt. A server enabled by the present invention could be 
farmed out these sites to handle high-traffic periods. 

A file system capable of handling many document 
requests also enables delivery of digital motion pictures and 
other content on demand using a computer network as a 
55 delivery system. In such a system, motion pictures are stored 
in time slice increments such as one minute intervals. A two 
hundred minute movie then consists of two hundred files, 
each of which can be retrieved from a server. Delivering a 
movie to many users would place a great load on one server, 
60 yet, by breaking the movie up into manageable chunks, it 
can be delivered effectively from multiple servers. By incor- 
porating the present invention into such a delivery system, 
the load of delivering the documents can be spread evenly, 
and with the possibility of adding and removing servers 
65 without reshuffling the entire file system. 

Other areas in which requests are allocated to resources 
that could benefit from the present invention include, but are 
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not limited to a user allocated to one of a plurality of data 19. The method of claim 1, wherein the step of mapping 

channels in a communications network, a telephone caller the request further comprises: 

allocated to one of a plurality of operators, and a physical /a dcriv ing a value from the request; and 

object allocated to one of a plurality of storage locations. ..... ... c * . ., 

It is felt, therefore, that these embodiments of the inven- s (u) „ dete ™ nin S * e out P u < ° f a funcUon ™* lhe value 

tion should not be limited to disclosed embodiments but d ™ Ved fr °" °f f ?~f aS , an mp " 1 - 

rather should be limited only by the spirit and scope of the 2 ° Jta me , lhod of *«» X < whet ™ lhe ste P of ma PP in g 

following claims eacn °* *" e pl urautv °* resources further compnses: 

We claim: (i) deriving a value for each resource; and 

1. A method for distributing a request to one of a plurality 10 (if) determining the output of a function using each 
of resources, comprising the steps of: derived value as an input. 

mapping said request to a request location in a math- 21. An apparatus for distributing a request to one of a 

ematical mapping space; plurality of resources, comprising: 

mapping each of said plurality of resources to a respective the pmrality of resources; 

at least one resource location in the mathematical 1C . . . 

mapping space; 1 an in P ut reccivin S a resource request; 

allocating said request to one of said plurality of resources a mapper in communication with said input, said mapper 
based on a mathematical relationship between the mapping said request to a request location in a math- 
request location in the mathematical mapping space ematical mapping space and mapping each of said 
and each of the resource locations in the mathematical plurality of resources to at least one resource locations 
mapping space. 20 in said mathematical mapping space; and 

2. The method of claim 1, further comprising before the a distributor distributing said request to one of said 
step of mapping the request the step of receiving a request. luralit of resources ^ response t0 a mathematical 

3. The method of claim 2, further comprising before the relationship between the request location in the math- 
step of mapping the request, the step of receiving a request ematical m and the at least one resource 
from a network node. 25 . .. , " . . ... c ... 

. tv _ ,, jri-ic^v. u c .u location of each of the plurality of resources in the 

4. The method of claim 3, further comprising before the . . . r 3 

step of mapping the request, the step of receiving a request ema C ma PP m 8 s P ace- .... 

from a network node, the network node comprising a web 22 ' ™* apparatus of claim 21, wherein the input recerves 

browser request from a network node. 

5. The method of claim 1 wherein the request is a data 30 ^ ^ apparatus of claim 22 wherein the input receives 
request and the resource is a server. a rK I ue * from a network node > the network node comprising 

6. The method of claim 5, wherein the request is a uniform a web browser. 

resource locator (URL) and the resource is a web server that A 24 - ^ apparatus of claimed 21, wherein the request is a 

can respond to URL request. data re 9 uest resources are servers. 

7. Trie method of claim 1 wherein the mathematical 31 £• ^ °f claun 24, wherein the request is a 
mapping space is a circle. umform l^tor (URL) and the resource is a web 

8. The method of claim 1 wherein the step of mapping server that can respond to a URL request, 
eachofsaidpluralityofresourcestoarespectiveatleastone 26 \ ™* apparatus of claim 21, wherein the step of 
resource location in the mathematical mapping space com- ma PP m 8 each of P lllraht y ° f further com- 
prises hashing each of said plurality of resources to the 40 P nses - 

respective resource location in the mathematical mapping (0 a vame generator for deriving a value for each 

space. resource; and 

9. The method of claim 1 wherein the step of mapping (ii) a function calculator for determining the output of a 
said request to a location in a mathematical mapping space function using each derived value as an input, 
comprises hashing the request to the request location in 45 27. The apparatus of claim 26, wherein the function 
mathematical mapping space. comprises a hash function. 

10. The method of claim 1, further comprising before the 28. The apparatus of claim 21, wherein the apparatus 
step of mapping the request, the step of generating a request. comprises a web cache. 

U. The method of claim 1, further comprising, after the 29. The apparatus of claim 21, further comprising a 

allocating step, the step of transmitting the request to the 50 request generator for generating the request, 

allocated resource. 30. The apparatus of claim 21, further comprising a 

12. The method of claim 1, further comprising, after the transmitter for transmitting the request to the allocated 
allocating step, the step of transmitting information describ- resource. 

ing the allocated resource to a network node. 31. The apparatus of claim 21, further comprising a 

13. The method of claim 1, further comprising, after the 55 transmitter for transmitting information describing the allo- 
allocaling step, storing the request and the information cated resource to a network node. 

describing the allocated resource. 32. The apparatus of claim 21, further comprising a data 

14. The method of claim 1, wherein the mapping space store for storing the request and the information describing 
comprises a line. the allocated resource. 

15. The method of claim 1, wherein the mapping space 60 33. The apparatus of claim 21, wherein the mapping space 
comprises a plane. comprises a circle. 

16. The method of claim 1, wherein the mapping space 34. The apparatus of claim 21, wherein the mapping space 
comprises a multi-dimensional mapping space. comprises a line. 

17. The method of claim 1, wherein the at least one 35. The apparatus of claim 21, wherein the mapping space 
resource location comprises at least two resource locations. 65 comprises a plane. 

18. The method of claim 1, wherein the request is a node 36. The apparatus of claim 21, wherein the mapping space 
and the resources are servers. comprises a multi-dimensional mapping space. 
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37. The apparatus of claim 21, wherein the al least one 
resource location comprises at least two resource locations. 

38. The apparatus of claim 21, wherein the request is a 
node and the resources are servers. 

39. The apparatus of claim 21, wherein the mapper further 5 
comprises: 

(i) a value generator for deriving a value from the request; 
and 

(ii) a function calculator for determining the output of a 
function using the value derived from the request as an 10 
input. 

40. The apparatus of claim 21, wherein the function 
comprises a hash function. 

41. A method for distributing a request for a web page in 
the form of a URL to one of at least one web servers, the 15 
method comprising the steps of: 

mapping the web page request to a mapping space by 
providing the web page request in the form of a URL 
as an input to a mapping function; 2Q 

mapping each of the at least one web servers to the 
mapping space; 

determining which of the at least one web servers is 
closest to the request in the mapping space; 
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directing the web page request to the web server deter- 
mined to be closest to the request in the mapping space. 

42. The method of claim 41, wherein the mapping space 
is at least one of a line, a circle, a plane, and a multi- 
dimensional space. 

43. The method of claim 41, wherein the web servers 
comprise web caching servers. 

44. A computer- readable medium whose contents cause a 
computer system to distribute a request to one of a plurality 
of resources, the computer system executing the contents of 
the computer-readable medium by performing a program 
comprising the steps of: 

mapping said request to a request location in a math- 
ematical mapping space; 

mapping each of said plurality of resources to a respective 
at least one location in the mathematical mapping 
space; 

allocating said request to one of said plurality of resources 
based on a mathematical relationship between the 
request location in the mathematical mapping space 
and the resource location of each of the plurality of 
resources in the mathematical mapping space. 

***** 
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